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WHAT IS CLAIMED IS: 

1 . A method of efficiently encoding a portion of a digital image frame 
comprised of image segments having arbitrary shapes, wherein each image segment has a 
boundary and each image segment is comprised of pixels having values, the method 
5 comprising: 

obtaining a first image frame and a second image frame; 

determining newly uncovered image segments in the second frame. 

determining which segments are adjacent to the newly uncovered image 
segment in the first image frame and designating them as boundary segments; 
1 0 carrying out a reference filling routine; 

determining which boundary segments are to be used for filling the exposed 
area and designating them as fill segments; 

carrying out a predictive filling routine; 

recalculating if necessary and determining the final set of fill segments. 
15 2. The method of carrying out a reference filling routine, the steps 

comprising: 

considering a region of adaptive dimensions around each unfilled pixel at the 
boundary of the exposed area; 

determining the statistical distribution of colors for all pixels within the region 
20 belonging to each boundary segment; 

3. The method of claim 2 further comprising determining to which 
segment each pixel is adjacent 

4. The method of claim 2 further comprising calculating a statistical 
parameter of the color values for each adjacent segment determined from the said statistical 

25 distribution of colors. 

5. The method of claim 2 further comprising calculating the difference 
between the actual color values of the pixel and the value of the calculated statistical 
parameter for each boundary segment and identifying the smallest difference value and the 
segment which provides the smallest difference value. 

30 6. The method of claim 2 further comprising filling the pixel with the 

value of the calculated statistical parameter of the segmentadjacent to the selected pixel that 
has the smallest difference if this difference is less than a threshold value. 
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7. The method of claim 2 further comprising assigning to the pixel, the 
segment identifier of the segmentadjacent to the selected pixel that has the smallest difference 
between the actual color value of that pixel and the calculated statistical parameter if this 
difference is less than a threshold value. 
5 8. The method of claim 2 further comprising leaving the pixel unfilled if 

the difference between the actual color value of that pixel and the calculated statistical 
parameter is greater than a threshold value. 

9. The method of claim 2 further comprising increasing the threshold 
value if a certain percentage or less of pixels can be filled with the current threshold. 
10 10. The method of claim 8 where the percentage of pixels is zero. 

1 1 . The method of claim 2 further comprising repeating the steps in claims 
2 through 7 for each unfilled pixel at the boundary of the exposed area until the entire 
exposed area is filled. 

12. The method of claim 2 where statistical distributions of three color 
15 components Y, U, V are determined. 

1 3 . The method of claim 2 where statistical distributions of each 
component of a multi-spectral image is determined. 

14. The method of claim 4 where the statistical parameter is the median 
color value of the statistical distribution. 

20 15. The method of claim 4 where the statistical parameter is any statistical 

moment of the distribution of color values. 

16. The method of determining the segments to be used for filling, the 
steps comprising: 

calculating the percentage of pixels within the exposed area that are 
25 filled by each of the boundary segments; 

calculating a parameter that represents the geometric shape of the area 
filled by each of the boundary segments. 

17. The method of claim 16 where the parameter representing the 
geometric shape is a function of the perimeter length, or a function of the area. 

30 18. The method of claim 16 where the parameter representing the 

geometric shape is a function of both the perimeter length and the area. 

1 9. The method of claim 1 6 further comprising selecting a set of tentative 
fill segments, the steps comprising: 
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selecting a segment as a tentative fill segment, if their contribution to the filled 
region is greater than a predetermined contribution of all of the boundary segments; 

selecting a segment as a tentative fill segment, if their contribution is less than 
the predetermined contribution but greater than a certain threshold value, and the geometric 
5 parameter of the region filled by this segment is within a threshold range; 

rejecting a segment as a tentative fill segment if one of the above two criteria 

are not met. 

20. The method of claim 1 9 where the predetermined contribution is the 
average contribution of all of the segments. 
10 21. The method of claim 1 9 where the predetermined contribution is any 

statistical parameter of all of the segments. 

22. The method of claim 1 5 further comprising calculating a function of 
the perimeter lengths that each of the tentative fill segments contribute to the perimeter of the 
exposed area. 

O 1 5 23 . The method of claim 22 where the function of the perimeter lengths is 

the normalized ratio of the squares of the perimeter lengths. 
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24. The method of claim 15 further comprising calculating a function of 
the areas of the regions contributed by the each of the tentative fill segments. 

25. The method of claim 24 where the function of the areas is the 
normalized ratio of the areas. 



26. The method of claim 1 5 further comprising calculating a function of 
the difference between the ratios. 
25 27. The method of claim 26 where the function of the difference between 

the ratios is the sum of the absolute values of the difference between the ratios. 
28. The method of claim 1 5 further comprising: 

repeating the steps in claims 15 through 26 after excluding the segment where 
the value obtained by subtracting the normalized area from the normalized length squared is 
30 the greatest; 

determining if the sum of the absolute values of the differences recalculated in 
this step is smaller than that obtained in the previous calculation of the same parameter. 
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29. The method of claim 1 5 farther comprising repeating the steps in claim 
1 5 until the sum of the absolute values of the difference is greater than that calculated during 
the previous recalculation. 



5 30. The method of claim 15 further comprising selecting the segments 

used to calculate the lowest sum of absolute values of the differences as the fill segments. 

3 1 . The method of carrying out a predictive fill routine, the steps 

comprising: 

considering a region of adaptive dimensions around each unfilled pixel at the 
10 boundary of the exposed area; 

determining the statistical distribution of colors for all pixels within the region 
belonging to each boundary segment; 

32. The method of claim 20 further comprising determining to which 
boundary segment each pixel is adjacent. 

15 33. The method of claim 20 further comprising calculating a statistical 

parameter of the color values for each adjacent segment determined from the said statistical 
distribution of colors. 

34. The method of claim 20 further comprising assigning to the pixel, the 
segment identifier of the segment that contributes the greatest number of pixels to the said 



f II 20 region of adaptive dimensions around the pixel. 



35. The method of claim 20 further comprising filling the pixel with the 
value of the statistical parameter of the segment that contributes the greatest number of pixels 
to the said region of adaptive dimensions around the pixel. 

36. The method of claim 20 further comprising repeating the steps in 
25 claims 1 8 through 22 for each unfilled pixel at the boundary of the exposed area until the 

entire exposed area is filled. 

37. The method of claim 20 where statistical distributions of three color 
components Y, U, V are determined. 

38. The method of claim 20 where statistical distributions each component 
30 of a multi-spectral image is determined. 

39. The method of claim 22 where the statistical parameter is the median 
color value of the statistical distribution. 

40. The method of claim 22 where the statistical parameter is any 
statistical moment of the distribution of color values. 
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41 . The method of determining whether the tentative fill segments chosen 
above sufficiently approximate the actual value of the exposed area, the steps comprising: 

calculating the value of a function of the actual values and the reference fill 
values for regions filled by each of the fill segments; 

calculating the value of a function of the actual values and the predictive fill 
values for regions filled by each of the fill segments; 

determining whether for any segment, the value obtained by subtracting the 
function of the actual values and the reference fill values, from the function of the actual 
values and the predictive fill values is above a threshold. 

42. The method of claim 30 where the function of the actual values and the 
reference fill values is the average of the absolute values of the color differences between the 
actual values and the reference fill values. 

43. The method of claim 30 where the function of the actual values and the 
predictive fill values is the average of the absolute values of the color differences between the 
actual values and the predictive fill values. 

44. The method of determining to undertake a recalculation, if for any 
segment or segments, the value obtained by subtracting the function of the actual values and 
the reference fill values, from the function of the actual values and the predictive fill values is 
above a threshold. 

45. The method of claim 33 where the function of the actual values and the 
reference fill values is the average of the absolute values of the color differences between the 
actual values and the reference fill values. 

46. The method of claim 33 where the function of the actual values and the 
predictive fill values is the average of the absolute values of the color differences between the 
actual values and the predictive fill values. 

47. The method of recalculating the fill segments and refilling, the steps 

comprising: 

rejecting the segment or segments if, for any segment, the value obtained by 
subtracting the function of the actual values and the reference fill values, from the function of 
the actual values and the predictive fill values is above a threshold. 

Repeating the steps described in claims 2 through 29 after excluding the 
rejected segment or segments. 
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48. The method of claim 36 where the function of the actual values and the 
reference fill values is the average of the absolute values of the color differences between the 
actual values and the reference fill values. 

49. The method of claim 36 where the function of the actual values and the 
5 predictive fill values is the average of the absolute values of the color differences between the 

actual values and the predictive fill values. 

50. The method of claim 29 where recalculation is carried out multiple 

times. 

5 1 . The method of determining the final set of fill segments, if no 

1 0 recalculation is done, the steps comprising designating the set of fill segments used for the 
predictive fill as the final fill segments. 

52. The method of determining the final set of fill segments, if a 
recalculation is performed, the steps comprising: 

calculating the value of a function of the first predictive fill values and the 
1 5 actual values of all of the pixels in the exposed area; 

calculating the value of a function of the recalculated predictive fill values and 
the actual values of all of the pixels in the exposed area; 

designating the set of fill segments used for the first predictive fill as the final 
fill segments, if the function of the recalculated predictive fill values and the actual values is 
20 greater than the function of the first predictive fill values and the actual values; 

designating the set of fill segments used for the recalculated predictive fill as 
the final fill segments, if the function of the recalculated predictive fill values and the actual 
values is less than the function of the first predictive fill values and the actual values. 

53 . The method of claim 4 1 where the function of the first predictive fill 
25 values and the actual values is the average of the absolute values of the color differences 

between the first predictive fill values and the actual values. 

54. The method of claim 41 where the function of the recalculated 
predictive fill values and the actual values is the average of the absolute values of the color 
differences between the recalculated predictive fill values and the actual values. 

30 55. The method of efficiently encoding exposed areas with multiple 

subregions, the steps comprising: 

Determining final fill segments for each of the exterior subregions. 

Calculating the value of a function that represents the error in the predictive 
filling for each external subregion. 



23 



If the function representing the error is less than a threshold value, then filling 
the exterior subregion. 

Determining again the identities of the exterior subregions which include the 
subregions that were not filled in the first pass as well as regions that were previously in the 
5 interior of the exposed area and previously had no boundary segments. 

Filling the exterior subregions using fill segments which may include the 
subregions that were filled in the previous step. 

56. The method of claim 44 further comprising repeating the steps in claim 
44 until all subregions have been filled. 
10 57 . The method of claim 44 where the function that represents the error in 

predictive filling is the average of the absolute value of the difference between the predictive 
fill values and the actual pixel color values. 

58. Transmission of the identities of the fill segments to a decoder 
p 59. Transmitting the order of filling and the fill segment identities to the 

It 15 decoder if the exposed area is divided into multiple subregions. 

H 60. Coupling the information of the fill segments with any set of residue 

encoder to improve local quality. 



ii . S 
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